Systems and methods for digital watermark localization based upon background foreground estimation

ABSTRACT

A system and method for localizing an area of interest likely containing a digital watermark is disclosed. Image frames may be segmented into multiple tiles. A pixel having the maximum grayscale or other value and a pixel having the minimum grayscale or other value in each tile may be identified. Maximum and minimum image maps may be generated from the image frame by replacing each tile with the respective maximum and minimum grayscale or other value pixels. A background map may be generated based on a moving average of the grayscale values of the pixels in the image maps. Foreground map(s) may be generated based on the difference of the values from the image maps to the background map. A region of interest may be determined based on the background and foreground maps and provided to a watermark decoder. Content contained in the digital watermark may be read.

BACKGROUND

Digital watermarking has been increasingly popular in tagging andtracking goods, especially in a retail environment. Digital watermarking(DWM) technology is based upon digitally embedding—i.e.watermarking-tags or any other type of identification information withinother images. For example, multiple copies of a barcode or othermachine-readable indicia may be digitally watermarked within images,texture, and/or text in a package containing a product. A digitalwatermark decoder may decode digital watermarks from images, texture,and/or text to retrieve the watermarked barcodes. The barcodes may beread by a barcode reader.

A digital watermark, by definition, is hidden from plain view.Furthermore, a digital watermark does not have a finite pattern as othertags, such as a barcode. Therefore, a watermark decoder has to analyzeeach part of an image frame to determine whether there is a digitalwatermark in that part of the image frame. Image processing iscomputation intensive, and therefore analyzing each and every part ofmultiple image frames is computationally not efficient. Such brute-forceanalysis requires a lot of computing power and time. Current decodestrategy for DWM decoding is based on static and fixed position forcandidate regions, which is not optimal and uses a lot of processingresources for each processed image frame. A time consuming decodingprocess may not be applicable in a retail environment, where thecheckout process has to be fast and efficient.

Therefore, conventional technology for digital watermark decoding may beslow and may not be viable in a retail environment. As such, asignificant improvement in the digital watermark decoding technology tomake it fast, efficient, and applicable in a retail and industrialenvironment is desirable.

SUMMARY

To make digital watermark decoding faster and more efficient,identifying areas within an image frame that may more likely contain adigital watermark (DWM) may be performed. To identify regions within animage frame that have a higher likelihood of including a digitalwatermark, background and foreground estimations to identify foregroundareas of higher activity, which may likely contain a digital watermark,may be used. A digital watermark decoder may analyze identifiedforeground areas for digital watermark decoding and not the entireimage, thereby significantly reducing the computational burden, andincreasing speed of identifying and reading digital watermarks.

One embodiment of a method of decoding a digital watermark may includesegmenting an image frame within a sequence of image frames intomultiple distinct tiles including multiple pixels. A background map maybe generated based on the tiles. Furthermore, a foreground map may begenerated based on the tiles. A region of interest for digital watermarkdecoding may be determined based on the background and foreground maps.A digital watermark at least partially located in the region of interestmay be decoded.

One embodiment of a system for decoding a digital watermark may includea communications unit configured to receive a sequence of image framesand a processing unit in communication with the communications unit. Theprocessing unit may be configured to segment an image frame within asequence of image frames into multiple distinct tiles each includingmultiple pixels. The processing unit may be configured to generate abackground map and a foreground map based on the tiles. The processingunit may further be configured to determine a region of interest fordigital watermark decoding based on the background and foreground maps,and decode a digital watermark at least partially located in the regionof interest.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments of the present invention are described indetail below with reference to the attached drawing figures, which areincorporated by reference herein and wherein:

FIG. 1A is an illustration of an illustrative retail checkoutenvironment in which a barcode scanner is used to image a checkout itemwith a digital watermark embedded on a label of the item;

FIG. 1B is a schematic of an illustrative barcode scanner configured toread DWMs on items;

FIG. 2 is an illustration of illustrative software modules configured tolocalize areas for digital watermarks based on background and foregroundanalyses according to principles described herein;

FIG. 3 is an illustration of an illustrative flow diagram of a digitalwatermark localization process based on background foreground analysisaccording to principles described herein;

FIG. 4A is an illustration of an illustrative image frame potentiallyincluding a digital watermark;

FIG. 4B is an illustration of an illustrative tiled image framepotentially including a digital watermark;

FIG. 4C is an illustration of illustrative maximum and minimum mapsgenerated from the illustrative tiled image frame;

FIG. 4D is an illustration of an illustrative background map extractionsbased on the minimum and maximum map sequences of FIG. 4C;

FIG. 4E is an illustration of an illustrative foreground map extractionbased on the background map extractions of FIG. 4D;

FIG. 4F is an illustration of an illustrative re-projection of highactivity blocks to the image frame in the image domain;

FIG. 5 is an illustration of an illustrative flow diagram for updatingforeground thresholds;

FIG. 6 is an illustrative flow diagram of an illustrative process ofgenerating background and foreground maps; and

FIG. 7 is an illustration of an illustrative checkout receipt containingan encoded and embedded digital watermark that is machine-readable.

DETAILED DESCRIPTION OF THE DRAWINGS

With regard to FIG. 1A, an illustration of an illustrative scanenvironment 100 a is shown. The scan environment 100 a, which may beretail checkout environment, warehouse environment, distributionenvironment, or any other environment, may include a scanner 102 and ascan item 106, such as a can of soup with a label affixed thereto. Thescanner 102 may be, for example, a barcode scanner, a laser scanner, alinear imager, a presentation scanner, a two dimensional area imager,and/or any other type of scanner. The scanner 102 may be in connectionwith back-end computing system (not shown), such as a retail computer,point of sale (POS), or any other type of computing system. Theconnection may be wired with a physical electrical connection betweenthe scanner 102 and the back-end computing system. Alternatively, theconnection may be wireless, where the scanner 102 is connected to theback-end computing system through a wireless communications technology,such as Bluetooth®, Wi-Fi, ZigBee®, and/or other type of radio frequency(RF) communications technology. Although the environment 100 a shows ahandheld scanner 102, this disclosure is not intended to be limited to ahandheld scanner, and any other type of stationary or portable scannershould be considered within the scope of this disclosure.

The scan item 106 may be any kind of product to be scanned by thescanner 102. For example, in a retail environment, the scan item 106 maybe a consumer product presented by a customer or operator at a checkoutcounter. In an industrial environment, the scan item 106 may be apackage moving along a conveyor belt or otherwise transported. Inoperation, illustrated as a field-of-view 104 of the scanner 102, anytype of electromagnetic wave, such as visible light, laser, infrared toilluminate the surface of the scan item 106 may be projected such thatthe surface can be scanned and analyzed. In another embodiment, thescanner 102 may scan the surface of the scan item 106 by capturing animage using ambient light and without projecting any type ofelectromagnetic wave or illumination.

With regard to FIG. 1B, an illustration of an illustrative scanenvironment 100 b is shown. The illustrative scan environment 100 b mayinclude a scanner 110, a computing unit 118, and a scan item 128. Thescanner 110 may include an illuminator 132 and an imager 130. The imager130 may include one or more optical components, such as lenses, mirrors,and/or otherwise, as understood in the art. The illuminator 132 may beassociated with an illumination source 114. The illumination source 114may generate any type of electromagnetic radiation, such as visiblelight, laser, infrared, and any other type illumination used toilluminate the scan item 128. The illuminator 132 may modify/modulatethe electromagnetic radiation generated by the illumination source 114to control the pattern and/or directionality of illuminating the scanitem 128. An illustrative illumination 134 is shown to be illuminatingthe scan item 128. It should be understood that the imager 130 may scanthe portion of the scan item 128 illuminated by the illuminator 132. Forinstance, in a case of visible light illumination, the imager may be acamera that captures images of the portions of the scan item 128illuminated by the illuminator 132. The imager 130 may be associatedwith an imaging sensor 112 that may digitize any type of image capturedby the imager 130 to generate image data 116. Reflection signals 136 maybe illuminated onto the imaging sensor 112 via the imager 130. In anembodiment, the imaging sensor 112 may operate as a global shutter CMOSsensor, and may have a 1280×1024 resolution (1.3 Mp). It should beunderstood that alternative imaging sensor configurations may beutilized.

The computing unit 118 may include a processing unit 124, anon-transitory memory 120, an input/output (I/O) unit 126, and a storageunit 122. The processing unit 124 may include one or more processors ofany type, where the processor(s) may receive raw image data or partiallyprocessed image data 116 from the imaging sensor 112. In someembodiments, the scanner may include a field programmable gate array(FPGA) (not shown), which may downsample image frames captured by theimager 130.

In a first illustrative downsampling, the FPGA or other processingdevice may divide an image frame into a plurality of non-overlappingtiles, identify the pixel with the maximum grayscale value in each tile,and replace each pixel in each respective tile with the maximumgrayscale value. In a second illustrative downsampling, the FPGA orother processing device may divide an image frame into a plurality ofnon-overlapping tiles, identify the pixel with the minimum grayscalevalue in each tile, and replace each pixel in each respective tile withthe minimum grayscale value. In other words, in both of the illustrativedownsampling, the pixels of each tile are replaced by a single pixel,thereby scaling down the image frame by the size of the tile. One havingordinary skill in the art should understand that the aforementioneddownsampling may covert an image frame from the image domain to a mapdomain. One having ordinary skill in the art should further understandthat although the downsampling is described as being performed by anFPGA, this process may be partially or fully performed by the processingunit 124 or any other processor.

The non-transitory memory 120 may be any type of random access memory(RAM) from which the processing unit 124 may access raw or processedimage data and write one or more processor outputs thereto. The I/O unit126 may handle communications with devices, such as the scanner 110, theInternet, and/or any other devices using one or more communicationsprotocols, as understood in the art. The storage unit 122 may storesoftware modules implementing one or more image processing and watermarkdecoding algorithms along with data being captured and processed.Although the computing unit 118 is shown as a single unit in theillustrative environment 100 b, one having ordinary skill in the artshould understand multiple computing devices, including one or moredistributed computers, may be used to accomplish the functionalitydescribed herein. Furthermore, one having ordinary skill in the artunderstands that there may be multiple layers of computer processing,that is, a low intensity computer processing may be conducted locally,and more complex computer processing may be conducted remotely, such ason the cloud.

In operation, and as further described with regard to FIGS. 4A-4F, theprocessing unit 124 may receive a sequence of image frames in the mapdomain or perform operations to convert the sequence of image framesfrom the image domain to the map domain. The map domain may containdownsampled image frames, where each tile may be replaced by arespective pixel with the maximum grayscale value or the minimumgrayscale value. From the map domain, the processing unit 124 may updatea background map, where the background map may contain a moving averageof each of the pixels. In calculating the moving average, the processingunit 124 may not have to keep up with the frame rate of the scanner 110.Rather, the processing unit 124 may recalculate the moving average everysecond, or every five seconds, and/or any other time interval dependingupon the accuracy and efficiency desired. The processing unit 124 maygenerate a foreground map while keeping up with the frame rate of thescanner 110. That is, the processing unit 124 may compute thedifferences between the grayscale values of the pixels for every imageframe in the map domain (also referred to as map domain image frame)with the background map. If a difference is above a first threshold butbelow a second threshold, the processing unit 124 may indicate that therespective pixel is in a low activity foreground. If the difference isabove the second threshold, the processing unit 124 may indicate thatthe respective pixel is in a high activity foreground. One havingordinary skill in the art should understand that grayscale values andcomparisons described in this embodiment and other embodimentsthroughout this disclosure is merely for the ease of explanation and inno way is intended to limit the scope of this disclosure. Theembodiments of this disclosure are equally applicable other forms ofdigitized images, such as RGB (red-green-blue) color model and CMYK(cyan-magenta-yellow-black) color model, certain markings or symbols, orotherwise.

The processing unit 124 may segment an image frame in the map domaininto multiple overlapping or non-overlapping blocks depending upon thedistance between the scanner 110 and the scan object 128. For eachblock, the processing unit 124 may calculate an activity score based onthe grayscale values of the pixels in the respective block. Theprocessing unit 124 may (i) identify one or more boxes with the highestactivity scores and (ii) map those boxes to the original image frame inthe image domain. The mapped area of the original image frame in theimage domain may be designated by the processing unit 124 as an area ofinterest likely containing a digital watermark. The processing unit 124may then execute digital watermark decoding libraries in the area ofinterest to decode digital watermark contained therein. If theprocessing unit 124 successfully decodes a digital watermark in theregion of interest, the processing unit 124 may update the first and thesecond thresholds for foreground extraction. If however, the processingunit 124 does not find or decode the digital watermark in the region ofinterest, the processing unit 124 may not update the first and thesecond thresholds. The blocks may be scanned or shifted horizontallyfrom one side to another within the foreground active area(s) to furthersearch for a DWM. Alternative search patterns by the blocks may beutilized.

With regard to FIG. 2, a block diagram illustrative software modules 200is shown. The illustrative software modules 200 may include a backgroundidentifier module 202, a foreground identifier module 204, a digitalwatermark localizer module 206, a digital watermark decoder module 208,a digital watermark reader module 210, and a barcode reader module 212.The aforementioned software modules may be executed by a processor 214.It should be understood that additional and/or alternative softwaremodules 200 may be utilized. Moreover, alternative combinations of thesoftware modules 200 may be utilized.

The background identifier module 202 may identify a background in asequence of image frames in the map domain. In some embodiments, thebackground identifier module 202 may receive sequence of image frames inthe map domain (or, map domain image frames) from a field programmablegate array (FPGA) embedded in a sensor capturing the sequence of imageframes. In other embodiments, the background identifier module 202 mayreceive the sequence of image frames in the map domain from anothersoftware module executer by the processor 214. Regardless of the source,the sequence of image frames in the map domain may be a maximum mapsequence, where the corresponding sequence of image frames in the imagedomain may have been segmented into multiple tiles and each tile beingreplaced by the pixel having the maximum grayscale value therein. Inaddition or in the alternative, the sequence of image frames in the mapdomain may be a minimum map sequence, where the corresponding sequenceof image frames in the image domain may have been segmented intomultiple tiles and each tile being replaced by a pixel having a minimumgrayscale value therein. The background identifier module 202 maymaintain a moving average of each pixel in the sequence of image framesin the map domain.

Moreover, the background identifier module 202, for each pixel, maymaintain a table with a predetermined number (for example, 256) ofgrayscale values. After each predetermined time interval (for example, 1second or 5 seconds), the background identifier module 202 may removethe oldest grayscale value from each table, add the newest grayscalevalue from the newest image frame in the map domain, and recalculate themedian value. The repeatedly calculated median value therefore mayrepresent a moving average that is updated after the predetermined timeinterval by the background identifier module 202. The median values ofthe pixels may form a background map, which may be used by theforeground identifier module 204 to generate the foreground map.

The foreground identifier module 204 may identify foreground in thesequence of image frames in the map domain. Unlike the backgroundidentifier module 202, the foreground identifier module 204 may keep upwith the frame rate and execute the foreground identification processfor every received map domain image frame. For every pixel in a mapdomain image frame, the foreground identifier module 204 may calculatethe difference in grayscale values of the pixel in the map domain imageframe and the corresponding pixel in the background map. If thecalculated difference is above a first threshold (second threshold>firstthreshold) but below a second threshold, the foreground identifiermodule 204 may indicate that the pixel may be in a low activityforeground. If the calculated difference is above the second threshold,the foreground identifier module 204 may indicate that the pixel is in ahigh activity foreground. If the pixel is determined to be in a lowactivity foreground region, a corresponding gray level may be used tovisually identify the low activity region. If the pixel is determined tobe in a high activity foreground region, a corresponding white level maybe used to visually identify the high activity region. Alternativecolors, symbols, characters, tints, or otherwise may be utilized todistinguish the low and high activity foreground regions, as well.

The digital watermark localizer module 206 may identify a region ofinterest likely to contain a digital watermark. More specifically, thedigital watermark localizer module 206 may segment a map domain imageframe into multiple overlapping or non-overlapping blocks. For eachblock, the digital watermark localizer module 206 may calculate anactivity score based on the grayscale values of the pixels within theblock. The digital watermark localizer module 206 may then select one ormore blocks with the highest activity scores as a region of interest.The digital watermark localizer module may reproject the region ofinterest from the map domain to the image domain.

The digital watermark decoder module 208 may operate in the image domainto decode any digital watermark in the region of interest identified bythe digital watermark localizer module 206. The digital watermark readermodule 210 may read any digital watermark decoded by the digitalwatermark decoder module 208. The barcode reader module 212 may read anybarcode or machine-readable indicia decoded by the digital watermarkdecoder module 208. In some embodiments, the barcode reader module 213may read a non-watermarked barcode scanned by a barcode scannerconnected to the processor 214. It should be understood that the term“barcode” may refer to any machine-readable indicia, such as a QR code.

One having ordinary skill in the art should understand that therespective functionality of the aforementioned software modules ismerely exemplary and similar functionality may be achieved by differentset of software modules. Furthermore, the software modules describedherein may achieve alternative and additional functionality, whichshould be considered to be within the scope of this disclosure.

With regard to FIG. 3, a flow diagram of a digital watermarklocalization process 300 is shown. The process 300 may begin at step302, where a processor may segment an image frame within a sequence offrames into tiles. In some embodiments, a field programmable gate array(FPGA) or other processing device may segment the image frame prior tofurther processing, as detailed below, by the processor. For each imageframe, the FPGA may generate a minimum map and a maximum map. Togenerate the maximum map, the FPGA may identify the maximum grayscalevalue in each tile, and replace the respective tile with a pixel of themaximum grayscale value. To generate the minimum map, the FPGA mayidentify the minimum grayscale value in each tile, and replace therespective tile with a pixel of the minimum grayscale value. Forexample, if the image frame is segmented into tiles of 16 pixels by 16pixels, the size of each of the maximum map and the minimum may besmaller than the image frame by a scale of 16×16. In some embodiments,the FPGA may select a value less than the minimum and the maximum map toreduce the “salt and pepper” noise, as understood in the art.

In step 304, the processor may generate a background map based on thetiles. In an embodiment, the background map may be based upon themaximum map. In another embodiment, the background map may be based uponthe minimum map. In yet another embodiment, the background map may bebased upon a combination of the minimum and maximum map. The backgroundmap may be based upon a moving average of the grayscale value of eachpixel in the maximum map and/or the minimum map. More specifically, theprocessor may maintain a table of the grayscale value of each pixel inthe maximum and/or the minimum map. Each time a new frame is received,the processor may update the table adding the grayscale value of thepixel and remove the oldest value of the pixel. After the table isupdated, the processor may determine the median value of the entries inthe table such that the median value represents a moving average of therespective pixel. The moving average therefore represents the backgroundpixels. An illustrative pseudocode for generating a background map isshown in TABLE I:

TABLE I Pseudocode for Generating a Background Map Every N new image { Rearrange all maps  For each map  { For each pixel of the map {  removeK-oldest value  insert newest value  compute median value  updatebackground map }  } }

In step 304, the processor may generate a foreground map based on thetiles. The processor may keep up with the frame rate to generate theforeground map. In other words, the processor may generate a foregroundmap based upon every image frame captured by the sensor unlike thebackground map, where the processor may update the foreground map everypredetermined time interval, such as one second or five seconds. Togenerate the foreground map, the processor may compute the differencebetween the grayscale values of every pixel in the image map and therespective pixels in the background map and compare the differenceagainst two thresholds. If the processor determines that the differenceis below a first threshold, the processor may indicate that the pixel isa part of the background. If the processor determines that thedifference is above the first threshold but below the second threshold,the processor may indicate that the pixel is part of a low activityforeground. If the processor determines that the difference is above thesecond threshold, the processor may indicate that the pixel is a part ofa high activity foreground. An illustrative pseudocode for generating aforeground maps is provided in TABLE II:

TABLE II Pseudocode for Generating Foreground Map For every new image  {For each position p {  Reset Foreground val fv(p)=0  For each map {compare map(p) with background value bv(p)If(abs(map(p)−bv(p))>th1(mapType, map(p) )) {fv(p)= max(fv(p),v1)}// lowactivity foreground If(abs(map(p)−bv(p))>th2(mapType, map(p) )) {fv(p)=max(fv(p),v2)}//high activity foreground }  } }

In step 308, the processor may determine a region of interest based onthe foreground and background maps. More specifically, the process maysegment a map domain image frame into a plurality of blocks. The blocksmay be overlapping in some embodiments and may be non-overlapping inother embodiments. For each of the blocks, the processor may calculateactivity values based on the grayscale values of the pixels contained inthe respective block. For example, the processor may aggregate thegrayscale values of the pixels within the respective block to generate adetermined activity value. The processor may select one or more highestactivity blocks as a region of interest in the map domain image frameand re-project the region of interest to the image domain original imageframe. The blocks may be used to scan or otherwise search the activityregions to identify the DWM or portion thereof. An illustrativepseudocode for determining the region of interest based on the blocks isshown in TABLE II:

TABLE III Pseudocode for Determining a Region of Interest Set Block Size= B Empty (Activity, Position) List L Every map row (or column) {  findhighest activity (A) block position (P)  add couple (A,P) to List L }Sort List L Take first H elements of List L

In step 310, the processor may decode a digital watermark located in theregion of interest by using one or more watermark decoding libraries, asunderstood in the art.

With regard to FIG. 4A, an image frame 400 of in a sequence of imageframes is shown. The image frame 400 may include a picture of an objectcaptured by a scanner, as described in the embodiments above. The objectmay be printed or a sticker label 401 on which a DWM may be included. Inan embodiment, a 32 pixel wide metadata side curtain 405 may be computedon-the-fly by the FPGA and included with the tiled image frame 402. Themetadata may be encoded with minimum and maximum gray values, amongstother metadata information.

With regard to FIG. 4B, a tiled image frame 402 containing multipletiles 404 a-404 n (collectively 404) is shown. In some embodiments, thetiles 404 may be distinct and non-overlapping. For each of the tiles404, a processor or a field programmable gate array (FPGA) (referred toas “processor” for brevity) may identify a pixel in each of the tiles404 with the maximum grayscale value and a pixel with minimum grayscalevalue. In other words, the processor may identify the local maxima andlocal minima for the grayscale values within each tile. In someembodiments, the processor may not identify the absolute minimum and/orthe absolute minimum and implement one or more filters to mitigatesensor noise or any other type of noise. The number of pixels withineach of the tiles 404 may be the same or different, and may be set basedon resolution of an image sensor or otherwise.

With regard to FIG. 4C, a maximum map 406 and a minimum map 408generated by the processor from the image frame 400 is shown. Themaximum map 406 may contain a pixel for each tile in the tile imageframe 402. In other words, each tile in the tile image frame 402 may bedefined as a pixel with the maximum grayscale value in the respectivetile to generate the maximum map 406. Similarly, each tile in the tileimage frame 402 may be replaced by a pixel with the minimum grayscalevalue in the respective tile to generate the maximum map 408. Therefore,each of the maximum map 406 and the minimum map 408 may be scaled downby the factor of the size of the tile in the tile image frame 402. Forexample, if the tile image frame 402 is 1024 pixels by 1024 pixels andthe tile is 16 pixels by 16 pixels, then the maximum map 406 may be(1024 pixels by 1024 pixels)/(16 pixels by 16 pixels), which is 64pixels by 64 pixels. This downsampling to generate the maximum map 406and the minimum map 408 may significantly decrease the computationalload in the downstream image processing in the subsequent steps.

With regard to FIG. 4D, an illustrative background map extraction/updateprocess is shown. The background map may be extracted/updated based on aminimum map sequence 408 and/or the maximum map sequence 410. For eachpixel the minimum map sequence 408, the processor may maintain a minimumpixel value table 412 and for each pixel in the maximum map sequence410, the processor may maintain the maximum pixel value table 414. Asshown herein, the size of the each of the minimum pixel value table 412and the maximum pixel value table 414 is five entries. However, onehaving ordinary skill in the art should understand that the minimumpixel value table and 412 and the minimum pixel value table 414 aremerely illustrative, and that the size of five entries is forillustrative purposes only. Typically, each of the minimum pixel valuetable 412 and the maximum pixel value table 414 may contain around 256entries. Other lengths of the tables 412 and 414 may be utilized.

The background map extraction process may be based upon the movingaverage of each pixel in the minimum map sequence 408 (to generate aminimum background map) and the maximum map sequence 410 (to generate amaximum background map). For example, for the pixel associated with theminimum pixel value table 414 in the minimum map sequence 408, theoldest grayscale value (13 as shown herein) from the oldest image framethe minimum map sequence 408 is discarded and the newest grayscale valueis added. As shown, the minimum map sequence 408 is darker than themaximum map sequence 410. After the addition of the newest grayscalevalue, a median of the grayscale value is computed for the minimum pixelvalue table. Similar operations may be performed for the maximum valuetable 414. The background map extraction process may not necessarily beperformed at the frame rate. Instead, the background map extractionprocess may be performed at exemplary time intervals, such as every onesecond to every five seconds. During the startup, the system may take upto two minutes to generate an initial background map. Other amounts ofthe time may be used based on a number of factors, such as frame rateand processor speed.

With regard to FIG. 4E, an illustrative foreground map extractionprocess is shown. The foreground extraction process may keep up with theframe rate, that is, the processor may extract a foreground map for eachimage frame captured by the sensor. The foreground map extractionprocess may be based upon a maximum map sequence and/or a minimum mapsequence, but for brevity, this process is described as being based on amap sequence 420, which may be a minimum map sequence and/or a maximummap sequence.

For each pixel in the map sequence 420 including maps 422 a, 422 b, 422c, 422 d, 422 e, 422 f, the processor may determine the differencebetween grayscale value of the pixel with the grayscale value of therespective background pixel, shown as black pixels. If the difference isabove a first threshold and below second threshold, the processor mayindicate that the pixel may be associated with a low activityforeground, shown as gray pixels. If the difference is above the secondthreshold, the processor may indicate the pixel may be associated with ahigh activity foreground, shown as white pixels. For example in theimage map 422 f, a first area 424 may include pixels associated with ahigh activity foreground and a second area 426 may include pixelsassociated with a low activity foreground.

With regard to FIG. 4F, an illustrative re-projection of high activityregions 428 a, 428 b, 428 c of the image map 422 f to the correspondingregions 430 a, 430 b, 430 c image frame 400 is shown. In other words,the FIG. 4F may illustrate a projection from the map domain to the imagedomain. The size of respective areas of the high activity regions 428 a,428 b, 428 c may be based upon various factors, such as the distance ofthe picture from the imager.

With regard to FIG. 5, an illustrative flow diagram of a process 500 forupdating a foreground threshold is shown. The process 500 may start atstep 502, where an image frame may be received. The image frame may becaptured by an image sensor or otherwise, as previously described. Instep 504, background maps may be extracted. Extracting the backgroundmaps may be performed using the same or similar process as describedwith regard to FIG. 4D, for example. In step 506, foreground maps may beextracted. Extracting the foreground maps may be performed using thesame or similar processes as described with regard to FIG. 4E. At step508, activity regions may be computed. Computing the activity regionsmay be performed using the same or similar processes as described withregard to FIG. 4F. The computed activity regions may be sorted. Sortingthe activity regions may be made based on size or area of the activityregions (e.g., white pixels as compared to gray pixels).

In step 512, a digital watermark decoder may be run on an original imagereprojection of high activity regions. If the digital watermark decodersuccessfully decodes a digital watermark in the high activity regions instep 514, the foreground thresholds may be updated in step 516.Otherwise, the foreground thresholds are not updated. After updating theforeground threshold maps, the process 500 may repeat starting from step506. An illustrative pseudocode for updating the foreground thresholdmaps is shown in TABLE IV:

TABLE IV Pseudocode for Updating Foreground Threshold Maps If(success) { for each map  { if( Activity of decoded Block > 4*th1(mapType, map(p))/(B*B)) {  increase th1 } if( Activity of decoded Block < 3*th1(mapType,map(p)) /(2*B*B)) {  decrease th1 } if( Activity of decoded Block >2*th2(mapType, map(p)) /(B*B)) {  increase th2 } if( Activity of decodedBlock < 3*th2(mapType, map(p)) /(2*B*B)) {  decrease th2 }  } }

With regard to FIG. 6, a flow diagram illustrating an illustrativeprocess 600 for generating foreground and background maps is shown. Instep 602, a minimum background map is generated based on a minimum mapsequence. More specifically, the minimum background map is generatedbased on the downsampling of the sequence of images by replacing tilesin the images with pixels with lowest grayscale value within therespective tiles. For every pixel, which is now the size of a tile, inthe downsampled sequence of images, a median grayscale value ismaintained, thereby forming a minimum background map. In step 604, amaximum is generated based on a maximum map sequence. More specifically,the maximum background map is generated based on the downsampling of thesequence of images by replacing tiles in the images with pixels withhighest grayscale value within the respective tile. For every pixel(i.e., size of a tile) in the downsampled sequence of images, a mediangrayscale value is maintained, thereby forming a maximum background map.Each of the background maps may be updated in predetermined timeintervals, such as 1 second or 5 seconds. In step 606, a low activityforeground map is generated. The low activity foreground map may includepixels, where the difference between the grayscale values of the pixelsand the respective pixels in a background map is above a first thresholdbut below a second threshold (e.g., gray color pixels). In step 608, ahigh activity foreground map is generated. The high activity foregroundmap includes pixels, where the difference between the grayscale valuesof the pixels and the respective pixels in the background map is abovethe second threshold (e.g., white color pixels). Each of the foregroundmaps may be generated at the frame rate.

In some embodiments, a foreground map may be segmented into multipleblocks. An activity score for each of the plurality of blocks may becalculated, and a block with the highest activity score may be selectedas a region of interest. The size of the plurality of blocks may beselected based on the distance between a scanner generate an image frameand an object in the image frame. Furthermore, the region of interestmay be projected into the original image frame.

FIG. 7 is an illustration of an illustrative checkout receipt 700containing an encoded and embedded digital watermark that ismachine-readable. Such a digital watermark is generally not perceptibleby a human eye, as understood in the art. As described in theembodiments above, the digital watermark may be hidden in the texture ofthe checkout receipt 700. The embedded digital watermark may be decodedby one or more embodiments described throughout this disclosure. Abarcode reader may read machine-readable data represented by the decodedDWM.

The foregoing method descriptions and the process flow diagrams areprovided merely as illustrative examples and are not intended to requireor imply that the steps of the various embodiments must be performed inthe order presented. As will be appreciated by one of skill in the artthe steps in the foregoing embodiments may be performed in any order.Words such as “then,” “next,” etc. are not intended to limit the orderof the steps; these words are simply used to guide the reader throughthe description of the methods. Although process flow diagrams maydescribe the operations as a sequential process, many of the operationscan be performed in parallel or concurrently. In addition, the order ofthe operations may be re-arranged. A process may correspond to a method,a function, a procedure, a subroutine, a subprogram, etc. When a processcorresponds to a function, its termination may correspond to a return ofthe function to the calling function or the main function.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the embodiments disclosedherein may be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and steps have been described above generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, but such implementation decisions should not beinterpreted as causing a departure from the scope of the principles ofthe present invention.

Embodiments implemented in computer software may be implemented insoftware, firmware, middleware, microcode, hardware descriptionlanguages, or any combination thereof. A code segment ormachine-executable instructions may represent a procedure, a function, asubprogram, a program, a routine, a subroutine, a module, a softwarepackage, a class, or any combination of instructions, data structures,or program statements. A code segment may be coupled to another codesegment or a hardware circuit by passing and/or receiving information,data, arguments, parameters, or memory contents. Information, arguments,parameters, data, etc. may be passed, forwarded, or transmitted via anysuitable means including memory sharing, message passing, token passing,network transmission, etc.

The actual software code or specialized control hardware used toimplement these systems and methods is not limiting of the invention.Thus, the operation and behavior of the systems and methods weredescribed without reference to the specific software code beingunderstood that software and control hardware can be designed toimplement the systems and methods based on the description herein.

When implemented in software, the functions may be stored as one or moreinstructions or code on a non-transitory computer-readable orprocessor-readable storage medium. The steps of a method or algorithmdisclosed herein may be embodied in a processor-executable softwaremodule which may reside on a computer-readable or processor-readablestorage medium. A non-transitory computer-readable or processor-readablemedia includes both computer storage media and tangible storage mediathat facilitate transfer of a computer program from one place toanother. A non-transitory processor-readable storage media may be anyavailable media that may be accessed by a computer. By way of example,and not limitation, such non-transitory processor-readable media maycomprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othertangible storage medium that may be used to store desired program codein the form of instructions or data structures and that may be accessedby a computer or processor. Disk and disc, as used herein, includecompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), floppy disk, and Blu-ray disc where disks usually reproduce datamagnetically, while discs reproduce data optically with lasers.Combinations of the above should also be included within the scope ofcomputer-readable media. Additionally, the operations of a method oralgorithm may reside as one or any combination or set of codes and/orinstructions on a non-transitory processor-readable medium and/orcomputer-readable medium, which may be incorporated into a computerprogram product.

The preceding description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, the present invention is notintended to be limited to the embodiments shown herein but is to beaccorded the widest scope consistent with the following claims and theprinciples and novel features disclosed herein.

The previous description is of a preferred embodiment for implementingthe invention, and the scope of the invention should not necessarily belimited by this description. The scope of the present invention isinstead defined by the following claims.

What is claimed:
 1. A method for decoding a digital watermark, themethod comprising: segmenting an image frame within a sequence of imageframes into a plurality of distinct tiles including a plurality ofpixels; generating a background map based on the tiles generating aforeground map based on the tiles; determining a region of interest fordigital watermark decoding based on the background and foreground maps;and decoding a digital watermark at least partially located in theregion of interest.
 2. The method according to claim 1, furthercomprising: reprojecting the region of interest to the image frame. 3.The method according to claim 1, wherein the background map is generatedfor predetermined time intervals, and wherein the foreground map isgenerated based on the frame rate of the sequence of image frames. 4.The method according to claim 1, wherein determining the region ofinterest comprises: segmenting the foreground map into a plurality ofblocks; calculating a activity score for each of the blocks; andselecting a block with the highest activity score as the region ofinterest.
 5. The method according to claim 4, further comprising:selecting a size for the plurality of blocks based on a distance betweena scanner generating the image frame and an object in the image frame.6. The method according to claim 1, wherein the tiles are in a grayscaleformat.
 7. The method according to claim 6, wherein generatingforeground map includes (i) generating a low activity foreground mapwherein the respective differences between the grayscale values of thepixels in the low activity foreground map are above a first thresholdand below a second threshold and (ii) generating a high activityforeground map, wherein the respective differences between the grayscalevalues of the pixels in the low activity foreground map are above thesecond threshold.
 8. The method of claim 7, further comprising: updatingthe first and second thresholds based upon receiving an indication of asuccessful decoding of a watermark in the region of interest.
 9. Themethod according to claim 6, wherein the background map includes aminimum background map based on a pixel with the minimum grayscale valuefor each of the plurality of tiles and generated by replacing by eachtile by the respective pixel with the minimum grayscale value.
 10. Themethod according to claim 9, wherein the background map includes amaximum background map based on a pixel with the maximum grayscale valuefor each of the plurality of tiles and generated by replacing by eachtile by the respective pixel with the maximum grayscale value.
 11. Themethod of claim 10, wherein the background map is generated by updatinga previously generated background map by replacing the respective oldestgrayscale values for one or more pixels of the previously generatedbackground map, adding the respective grayscale values for the one ormore pixels, and calculating the median grayscale values for the one ormore pixels such that the background map represents a moving average ofthe grayscale values of the one or more pixels.
 12. A system of decodinga digital watermark, the system comprising: a communications unitconfigured to receive a sequence of image frames; a processing unit incommunication with the communications unit, and configured to: segmentan image frame within a sequence of image frames into a plurality ofdistinct tiles including a plurality of pixels; generate a backgroundmap based on the tiles; generate a foreground map based on the tiles;determine a region of interest for digital watermark decoding based onthe background and foreground maps; and decode a digital watermark atleast partially located in the region of interest.
 13. The systemaccording to claim 12, wherein the processing unit is further configuredto: segment the foreground map into a plurality of blocks; calculate aactivity score for each of the blocks; and select a block with thehighest activity score as the region of interest.
 14. The systemaccording to claim 12, wherein the processing unit is further configuredto: generate the background map for predetermined time intervals; andgenerate the foreground map at the frame rate of the sequence of imageframes.
 15. The system according to claim 12, wherein the tiles are in agrayscale format.
 16. The system of claim 15, wherein the processingunit is further configured to generate the background map by updating apreviously generated background map by replacing the respective oldestgrayscale values for one or more pixels of the previously generatedbackground map, adding the respective grayscale values for the one ormore pixels, and calculating the median grayscale values for the one ormore pixels such that the background map represents a moving average ofthe grayscale values of the one or more pixels.
 17. The system accordingto claim 15, wherein the background map includes a minimum backgroundmap based on a pixel with the minimum grayscale value for each of theplurality of tiles and generated by replacing by each tile by therespective pixel with the minimum grayscale value.
 18. The systemaccording to claim 17, wherein the background map includes a maximumbackground map based on a pixel with the maximum grayscale value foreach of the plurality of tiles and generated by replacing by each tileby the respective pixel with the maximum grayscale value.
 19. The systemaccording to claim 15, wherein the processing unit is further configuredto: generate a low activity foreground map wherein the respectivedifferences between the grayscale values of the pixels in the lowactivity foreground map are above a first threshold and below a secondthreshold; and generate a high activity foreground map wherein therespective differences between the grayscale values of the pixels in thelow activity foreground map are above the second threshold.
 20. Thesystem according to claim 19, wherein the processing unit is furtherconfigured to: update the first and second thresholds based uponreceiving an indication of a successful decoding of a watermark in theregion of interest.